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PREFACE 

In  the  operations  analysis  curriculum  at  the  United  States  Naval 
Postgraduate  School  numerous  reports  were  presented  from  1952  to  1954 
which  exemplified  some  methods  and  techniques  for  comparing  weapons, 
methods,  strategies,  etc.,  quantitatively  using  a  measure  of  effective- 
ness. These  reports  seemed  to  place  little  emphasis  on  the  relative 
amount  of  effort  or  assets  available  to  implement  a  final  decision. 
This  paper  is  written  to  emphasize  the  necessity  of  considering  not 
only  the  amount  of  effort  available  but  also  the  manner  and  times  at 
which  increments  of  effort  become  available. 

The  writer  would  like  to  express  sincere  appreciation  to  Drs.  T.E. 
Oberbeck  and  C.C.  Torrance  of  the  Naval  Postgraduate  School  for  their 
guidance  and  criticism  during  the  preparation  of  this  paper.  Dr.  Ober- 
beck spent  countless  hours  examining  the  programming  aspects  particularly 
and  provided  invaluable  assistance  in  the  phrasing  and  wording  of  dif- 
ficult sections.  Dr.  Torrance  suggested  the  addition  of  illustrative 
problems  to  demonstrate  the  use  of  the  theoretical  concepts  in  practi- 
cal situations  and  helped  reword  obscure  sections  throughout.  Neither 
should  be  held  accountable  for  other  obscure  passages  which  undoubtedly 
remain. 
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CHAPTER  I 
INTRODUCTION 

Measures  of  effectiveness  such  as  exchange  rates,  search  rates,  loss 
rates,  production  rates,  transportation  rates,  and  other  ratio  criteria, 
can  be  used  effectively  to  make  a  quantitative  comparison  of  alternative 
methods  of  accomplishing  a  given  objective.  Writers  of  practicability 
studies  and  feasibility  studies  compare  proposed  weapons,  strategies,  de- 
vices, and  methods  using  measures  of  effectiveness.  Military  leaders  and 
civilian  executives  use  measures  of  effectiveness  either  consciously  or 
unconsciously  when  seeking  a  quantitative  basis  for  decision.  This  paper 
introduces  three  considerations  that  affect  the  use  of  measures  of  ef- 
fectiveness and  presents  representative  problem  solutions  which  demon- 
strate methods  of  dealing  with  each. 

The  considerations  mentioned  above  are  the  basis  of  three  types  of 
problems : 

(1)  Optimum  Distribution  of  Effort  Problems. 

(2)  Linear  Programming  or  Activity  Analysis  Problems. 

(3)  Game  Theory  Problems. 

Each  of  these  problems  involves  restrictions  that  may  be  indicated  as 

follows : 

Optimum  Distribution  of  Effort  Problem. 

A  limited  amount  of  money,  manpower,  materials,  or  other  assets  is 
available  to  accomplish  a  given  task.  Two  or  more  "alternatives"  (methods, 
devices,  strategies,  or  weapons)  are  to  be  compared  on  the  basis  of  that 
limited  amount.  For  example,  suppose  an  executive  of  a  hypothetical  steel 
mill  faces  the  following  situation.  An  urgent  order  is  received  for 


Hi-chrome  steel  (a  hypothetical  type  of  steel).  Furnaces  number  one  and 
two  which  can  produce  Hi-chrome  steel  are  shut  down.  Only  a  limited  sup- 
ply of  coal  is  stockpiled,  and  it  is  apparent  that  no  more  coal  can  be 
obtained  before  the  time  limit  expires  for  the  steel  order.  Furnace  num- 
ber one  requires  relatively  little  coal  during  warm-up  and  reaches  its 
optimum  usage  ratio  (defined  as  the  number  of  tons  of  steel  produced  per 
ton  of  coal  consumed)  soon  after  warm-up  is  completed.  Furnace  number 
two  requires  more  coal  for  warm-up  and  reaches  its  optimum  usage  ratio 
more  slowly  than  number  one,  but  has  a  better  optimum  usage  ratio  than 
number  one.  The  steel  executive  desires  to  manufacture  as  much  Hi-chrome 
steel  as  possible,  using  the  available  coal  regardless  of  all  other  con- 
siderations, and  he  has  the  following  alternatives:  (1)  operate  number 
one  alone,  (2)  operate  number  two  alone,  or  (3)  operate  both  simul- 
taneously. See  Appendix  for  a  problem  solution  of  this  type. 
Linear  Programming  Problems. 

A  limited  amount  of  some  asset,  say  steel,  is  now  available  to  ac- 
complish a  given  task,  but  the  quantity  of  available  steel  may  be  in- 
creased by  a  process  involving  feedback  of  some  of  the  steel,  although 
the  process  is  time  consuming.  The  given  task  requires  the  maximization 
of  some  quantity  at  the  end  of  a  specified  time  interval  subject  to  a 
system  of  inequalities.  The  maximization  process  might  be  static  or  dy- 
namic. If  static,  the  feedback  ratio  is  initially  determined  and  re- 
mains constant  thereafter.  If  dynamic,  the  feedback  ratio  is  changed 
continuously  or  as  required  to  produce  an  optimum  solution.  As  an  exam- 
ple of  dynamic  programming,  consider  a  steel  executive  with  an  initial 
amount  of  steel  on  hand  and  an  initial  number  of  furnaces  who  desires  to 


maximize  his  steel  stockpile  at  the  end  of  a  specified  time  interval. 
He  must  allocate  some  fraction  of  the  available  steel  to  building  new 
furnaces,  another  fraction  to  be  exchanged  for  the  raw  materials  re- 
quired in  making  new  steel,  and  the  remaining  fraction  to  be  stockpiled. 
Any  one  or  two  of  these  fractions  may  be  zero  as  long  as  all  of  the  steel 
is  devoted  to  one  of  the  three  uses;  and  the  above  fractions  may  be 
changed  continuously  or  as  desired.  Assume  that  the  rate  of  increase  in 
the  number  of  furnaces  is  linearly  dependent  on  the  quantity  of  steel 
devoted  to  making  furnaces  and  the  rate  of  steel  production  is  linearly 
dependent  on  the  quantity  of  steel  being  exchanged  for  the  raw  materials 
and  services  required  to  make  new  steel.  The  executive  must  find  a  sys- 
tem for  allocating  available  steel  so  as  to  maximize  the  amount  of  stock- 
piled steel  at  the  end  of  the  specified  time  interval.  Chapter  III  shows 
a  solution  to  a  particular  problem  of  this  type. 
Game  Theory  Problem. 

The  amount  of  effect  expected  from  a  unit  of  effort  in  one  of  the 
above  type  problems  depends  on  count ermea sure s  employed  by  some  opposing 
force.  For  example,  consider  a  state  of  war  to  exist  when  the  steel  ex- 
ecutive in  the  first  example  above  receives  the  order  for  Hi-chrome  steel. 
Friendly  intelligence  reports  show  that  the  enemy,  knowing  the  strategic 
value  of  Hi-chrome  steel,  is  willing  to  expend  an  atomic  bomb  to  halt  as 
much  steel  production  as  possible.  The  furnaces  will  be  partially  opera- 
ble if  hit  by  a  bomb  and  their  usage  ratios  will  differ  as  stated  earlier. 
Assume  that  all  available  coal  must  be  stored  in  two  barges,  each  holding 
half  of  the  coal.  Further  assume  two  coal  storage  locations  for  the 
barges  near  each  furnace  site  such  that  a  bomb  hitting  near  either  site 


will  destroy  a  coal  barge  which  might  be  at  one  location,  but  will  not 
reach  barges  at  both  locations  simultaneously.  The  steel  executive  must 
decide  how  to  allocate  the  coal  so  as  to  maximize  the  expected  steel  pro- 
duction if  the  enemy  uses  an  optimal  strategy  in  deciding  which  factory 
to  bomb.  See  Appendix  for  a  problem  solution  of  this  type. 

The  preceding  types  of  problems  might  be  encountered  by  a  military 
commander  in  the  following  elementary  examples: 

(1)  Search  Problem.  A  downed  pilot  is  known  to  be  in  one  of  two 
separated  areas.  Two  search  planes  are  available  for  a  few  hours  only, 
and  a  decision  must  be  made:  (a)  to  send  both  planes  for  a  complete 
search  of  one  area  and  partial  search  of  the  other  area;  (b)  to  let  both 
planes  cover  both  entire  areas  by  flying  a  less  efficient  search  pattern; 
or  (c)  to  let  planes  split  up,  one  searching  each  area.  Data  is  available 
to  show  the  relative  efficiency  of  search  coverage  under  each  plan.  The 
vital  information  needed  here  is  the  number  of  hours  the  planes  will  be 
available  compared  with  the  size  of  the  areas  to  be  searched.  Other 
phases  of  the  problem  will  not  be  considered  by  the  methods  of  this  paper. 

(2)  Training  Problem.  A  new  type  of  airborne  detection  device  has 
proved  to  be  far  more  effective  than  previous  devices  in  locating  enemy 
submarines,  and  the  devices  are  plentiful;  but  very  few  operators  know 
how  to  use  them.  A  friendly  convoy  must  sail  in  a  few  days;  and  it  is 
desired  to  search  as  much  as  possible  of  the  proposed  convoy  route  with 
this  device  before  the  convoy  sails  to  give  our  anti-submarine  ships  a 
chance  to  engage  all  enemy  submarines  contacted  along  the  route.  It  takes 
time  for  friendly  ships  to  proceed  to  a  given  contact  and  engage  the  sub- 
marine; but  once  engaged  the  submarine  will  no  longer  be  considered  a 


threat  to  the  convoy.  It  Just  takes  a  few  hours  to  train  new  operators, 
but  only  a  limited  number  of  men  can  be  instructed  by  any  one  operator  at 
one  time;  and  an  operator  cannot  be  searching  and  training  other  operators 
simultaneously.  The  military  commander  must  decide  how  to  assign  the 
trained  operators  between  searching  and  training  other  operators  so  as  to 
have  pre-searched  the  greatest  possible  part  of  the  proposed  convoy  route 
before  the  convoy  sails. 

(3)  Training  Problem  with  Attrition.  This  problem  includes  all  ele- 
ments of  the  above  problem  plus  the  following:  The  enemy  knows  the 
friendly  military  commander's  problem  and  tries  to  defeat  his  purpose  by 
either  (a)  concentrating  his  efforts  against  the  search  planes,  or  (b) 
concentrating  against  the  training  facilities. 

This  paper  will  introduce  simple  examples  of  the  above  types  of  prob- 
lems to  show  some  techniques  and  reasoning  which  may  be  useful  in  dealing 
with  the  considerations  given  above.  Chapter  II  deals  with  the  first  type 
problem,  Chapters  III  and  IV,  with  the  second  type,  and  Chapter  V,  very 
briefly  with  the  third  type  of  problem.  The  second  and  third  types  are 
logical  extensions  of  the  first  type,  and  the  third  type  may  be  an  exten- 
sion of  the  second  type  in  many  instances. 


CHAPTER  II 
OPTIMUM  DISTRIBUTION  OF  EFFORT 

Dr.  Bernard  Koopmauv  '  proposed  a  graphical  method  and  an  analyti — 
cal  method  of  finding  the  optimum  distribution  of  effort  under  certain 
conditions  when  a  limited  amount  of  effort  is  available  for  the  perfor- 
mance of  two  related  tasks.  It  would  appear  that  the  graphical  method 
is  sufficient  to  solve  problems  under  the  assumptions  given  below;  but 
it  is  believed  that  a  brief  examination  of  some  representative  analyti- 
cal solutions  will  help  to  establish  a  better  mental  picture  of  real-life 
problems  and  even  enable  the  reader  to  surmise  a  near-optimum  solution 
when  time  does  not  permit  a  thorough  study. 

The  following  assumptions  hold  throughout  this  chapter:  (l)  effort 
and  effect  can  be  expressed  numerically,  i.e.,  a  quantitative  basis  for 
decision  is  plausible;  (2)  effects  from  two  or  more  alternatives  (defined 
here  as  methods,  weapons,  devices,  or  strategies)  are  additive;  and 
(3)  the  effect  produced  by  a  given  effort  depends  only  on  the  amount  of 
that  effort  expended  and  the  alternative  to  which  the  effort  is  supplied; 
i.e.,  for  each  alternative  there  is  a  function  f^(x)  defined  as  a  return 
function  such  that  fi(x^)  is  the  effect  of  x^^  units  of  effort.  By  defini- 
tion, a  joint  return  function  E(x^,  x2)  is  the  sum  of  two  or  more  return 
functions:  i.e.,  E(x^,  x^)  «  fi(xi  )-f-f2(x2) »  where  x^-r  ^  =  c,  and  c  is 
the  total  effort  available. 

The  following  procedure  applies  when  finding  an  optimum  distribution 
of  effort  between  two  alternatives  by  graphical  methods.  (Reference  ^  des- 
cribes a  graphical  method  for  dealing  with  three  alternatives  in  two  di- 
mensions. The  extension  of  the  method  of  this  chapter  to  the  case  of 


three  alternatives  will  be  evident  from  a  consideration  of  the  techniques 
of  reference  4) • 

(1)  Construct  a  horizontal  line  segment  of  length  c  *  x-j_-r  Xp. 

(2)  At  each  end  of  this  line  erect  a  vertical  axis.  Label  the  left 
axis  the  f^(x)  axis  and  the  right  one  the  fAx)   axis. 

(3)  Since  any  point  on  the  horizontal  line  segment  divides  it  into 
two  smaller  segments  whose  sum  is  c,  let  these  smaller  segments  represent 
values  for  x.  and  x«.  Then  x_-r-x2  =  c.  Call  the  left  segment  x-,  and  the 
right  segment  x2#  Consider  only  points  on  the  horizontal  segment  which 
are  between  the  vertical  axes  since  negative  values  of  x  are  not  defined. 

(4)  Plot  two  points  directly  above  each  point  on  the  horizontal  line 
for  which  values  of  the  f^(x)  are  known,  one  point  to  represent  f,  (x^) 
and  the  other  to  represent  f^lx^).  Presumably,  the  return  functions  are 
not  known  implicitly. 

(5)  Join  the  above  points  by  smooth  curves  representing  f^(x)  and 
f2(x)  and  then  plot  a  third  curve,  E(xltx2),  by  summing  the  ordinates  of 
the  other  two  curves.  Figure  3,  page  10,  shows  several  representative 
plots  for  the  particular  return  functions  given  there,  and  these  plots 
will  be  discussed  later.  A  maximum  point  on  the  curve  of  E(x^,  Xg)  indi- 
cates the  maximum  effect  which  may  be  achieved  from  an  amount  of  effort, 
c,  by  using  the  two  given  alternatives  under  the  assumptions  given  at  the 
start  of  this  chapter. 

Since  we  seldom  expect  to  find  return  functions  which  satisfy  the 
third  basic  assumption,  it  is  convenient  to  define  a  conditional  return 
function.  ^(xp/f-,),  as  the  amount  of  additional  effect  obtainable  from 
x2  units  of  effort  applied  to  the  second  alternative  after  x^  units  of  ef- 
fort have  produced  an  effect  MO  from  the  first  alternative.  If  this 


conditional  return  function  is  substituted  for  f2^x2^  in  the  8tePs  of  the 
above  paragraph,  a  graphical  solution  showing  the  maximum  effect  possible 
may  be  obtained  even  though  the  original  return  functions  were  not  additive. 
(If  f1(x1)+  fgC^/f _)^  f^x^-f-f  (xj/f  ),  additional  analysis  is  required, 
but  such  an  analysis  is  not  desirable  here). 

For  his  analytical  solution  Dr.  Koopman^6'  chose  functions  which  can 
be  used  to  approximate  typical  behaviors  for  the  return  functions  by  vary- 
ing the  values  of  the  constants  involved  as  shown  in  Table  I  and  Figures  1 
and  2.  (Please  fold  out  page  \0   for  the  remainder  of  this  discussion). 
His  reasons  for  choosing  these  functions  were  given  as  follows: 

(1)  they  are  extremely  simple  and  amenable  to  mathematical  manipula- 
tion; 

(2)  they  are  immediate  generalizations  of  formulas  valid  in  actual 
cases;  and 

(3)  they  correspond  to  simple  quantitative  ideas;  i.e.,  they  can  be 
derived  from  simple  concepts  of  the  way  in  which  effort  produces  effect. 
To  get  an  optimum  solution  using  two  of  these  functions,  we  may  set  the 
first  and  second  derivatives  with  respect  to  x  of  the  joint  return  func- 
tion equal  to  zero  and  solve  for  a  maximum  point  by  elementary  calculus, 
checking  to  see  if  the  values  of  the  joint  return  function  at  the  end  points 
of  the  horizontal  segment  exceed  any  internal  maxima. 

By  examining  typical  plots  of  these  functions  such  as  are  shown  in 
Figures  1,  2  and  3,  we  may  classify  the  functions  in  one  of  the  following 
categories : 

(1)  Type  which  will  show  an  effect  proportional  to  the  effort  expended 
until  saturation  effects  occur  as  in  Figure  l(a)  and  2(a); 


(2)  Type  which  will  require  either  moat  of  the  available  effort  or 
else  none  at  all  (see  Figure  l(b); 

(3)  Type  which  will  require  at  least  an  amount  of  effort  sufficient 
to  produce  a  near-saturation  effect  or  else  none  at  all  (see  Figure  2(b); 
and  (i+)  Type  which  should  always  be  used  (but  to  a  relatively  small  ex- 
tent if  type  (2)  or  (3)  is  also  employed.  See  Figures  l(c)  and  2(c). 

A  thorough  study  of  the  possible  pairwise  combinations  of  the  func- 
tions given  in  Table  I  should  be  of  great  help  to  the  operations  analyst. 
The  military  commander,  who  must  make  snap  decisions  in  combat  involving 
distribution  of  effort,  might  find  that  an  understanding  of  the  basic 
concepts  given  above  is  invaluable. 
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CHAPTER  III 
DYNAMIC  LINEAR  PROGRAMMING,  EXAMPLE  I 

The  preceding  problem  becomes  more  general  if  the  limited  assets 
available  vary  with  time  and  with  the  manner  in  which  we  have  expended 
previously  held  assets  and  if  we  desire  to  maximize  a  total  effect  at  the 
end  of  a  specified  time.  This  new  problem  is  a  programming  or  activity 
analysis  problem.  An  attempt  to  discuss  such  problems  in  general  would 
be  presumptuous  here;  but  a  simple  type  of  dynamic  linear  programming 
problem  will  be  introduced  in  this  chapter  and  then  expanded  in  the  fol- 
lowing chapter  to  present  some  methodology  and,  it  is  hoped,  some  concepts 
which  will  be  useful  in  dealing  with  measures  of  effectiveness.  The  fol- 
lowing problem  is  identical  mathematically  with  that  of  Dr.  Isaacs^-5', 
which  is  discussed  in  the  next  chapter. 

A  sailing  ship  is  in  an  ocean  bounded  by  infinite  stretches  of  land 
to  the  west  and  south  (see  Figure  4)  •  The  winds  over  this  ocean  are  al- 
ways westerly.  Wind  force  varies  with  position  but  not  with  time,  and 
the  force  at  any  given  spot  of  the  ocean  is  known.  The  ship  cannot  sail 
closer  than  90  degrees  into  the  wind. 

The  captain  of  this  ship  is  ordered  to  travel  as  far  north  from  his 
present  position  as  possible  by  the  end  of  a  specified  period.  Since  the 
wind  provides  the  only  power  for  propulsion,  the  captain  must  plan  his 
voyage  carefully  to  take  advantage  of  the  most  favorable  winds.  The  more 
he  travels  east,  the  stronger  the  winds  and  thus  the  faster  he  may  travel; 
but  an  easterly  component  of  his  velocity  reduces  the  northerly  component. 

The  captain,  being  something  of  a  mathematician  as  well  as  being  a 
good  sailor,  finds  that  he  can  express  the  speed  he  is  able  to  make  on  a 
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given  heading  by  using  the  following  mathematical  expressions: 
Let  x  *  number  of  miles  east  of  the  land  to  the  west. 
y  -   number  of  miles  north  of  the  land  to  the  south. 
x  s  east-west  component  of  velocity. 
f  ~   north-south  component  of  velocity. 
H  =  instantaneous  heading  of  the  ship.  (0  <  H  <  180) 

since  the  ship  cannot  head  closer  than  90°  into  the  wind. 
C  =  instantaneous  true  course  of  the  ship.  (Unfortunately, 

there  is  very  little  correlation  between  C  and  H  in  some 
cases). 
T0  ■  specified  time  in  which  the  captain  must  maximize  y. 
T  =  problem  time  remaining  at  time  t. 
h  =  a  number  representing  the  sailing  capabilities  of  the  ship. 

It  is  adjustable  within  certain  limits  which  are  dependent 

on  heading  and  position  as  follows: 

O  <  h  £    l&O-H  (1) 

and  k    <      fl&O  (2) 

H 

Then,  X   -   Jqq  y  (3) 

y--  'o(^y-(iw)y-(^o)y 

Due  to  shoaling  conditions,  x  >  1. 

The  ocean  may  be  divided  into  two  regions,  I  and  II,  as  shown  in 
Figure  U,   by  the  line:      y  «  x  (6) 
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In  region  I,        0  <   h  f^  180  -   H  (l) 

In  region  II,       0  <  h  ^  180  -  H  (l) 

and  n  —  y  180#  ^ 

We  notice  that  a  line  in  region  II  drawn  through  the  origin,  (0,0), 
with  a  slope,  m,  will  divide  the  ocean  into  two  parts  such  that 

0  <  h  ^Z.  i£2  for  all  positions  north  of  this  line.  (7) 

m 

•  • 

We  further  note  that       C  =  tan"1  sL    or  letting  c  ■•*-      (c  is  the  slope 
of  the  ship's  velocity  vector)         G  -  tain     C    . 


-U^r      =^-H 


then  e  -  *  =  ^OTT  ^       =  7n  '  "  (8) 


X  H        V 


and  also  cH  =  9h  -  H 


H 


or 


H  =  -2iL 


C*-l 


and  h  a  i-ULH  (9) 

9 

U)  may  be  written  y  =  -2^—  y  (10) 

180 

If,  momentarily,  we  require  that  h  -f-  H  =  180  (11) 

then  H  +    <°*1)H  -  180 

H  ^  =180 
9 

1620 
H  =  cTlO  (13) 

c  =  lf--10  (U) 

Also,  from  (11)  and  (13) 

1620 

h  -f  =  180 

c-f-10 

h  =  180  g±-  (15) 
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or 

180  -  lOh 
c  "   h  -  180 

and  from  (10), 

(11),  and  (12) 

.  _  9c 

y  -  „.-,n  y 

(15') 


(16) 


From  (l),  we  see  that   H  «  0  =>•  x  =  0 

and  thus,  when  we  start  from  any  point  P0(x0,  y0),  x(t)  =  x© 

for  all  t  as  long  as  H  =  0,  and  c  *  00 

From  (4)  we  see  that  the  maximum  value  obtainable  for  y  is  y  =  9y 

when    H  =  0  and  h  =  180. 

Since  these  conditions  may  be  satisfied  at  all  positions  in  region  I,  we 

should   choose  H  =  0  and  h  ■  180  as  long  as  the  ship  remains  in  region  I, 

This  gives:  y  =  y0e9t   (l  ^  y  <  x)  (17) 

In  region  II,  when  c  =  o=>  (and  H  =  0) ,  y  =  ~^  y  =  ^q"  y      (18) 

If  we  let  h  be  as  large  as  possible  to  maximize  y,  we  have: 

* =  20  y  =  O7 180)  20 =  ^  (2)  &  (18) 

and  y  =  y0  +■  9xQt  (for  H  =  0,  c  =  <=«  ,  and  h  =  -^°-  18o\       (19) 

Going  back  to  (18),  we  see  that  the  problem  here  is  to  make 
h  =  h(t)  as  large  as  possible  over  a  given  time  interval,  TQ,  in  which  we 
seek  to  reach  the  largest  possible  terminal  value  of  y,  remembering  that 
it  might  be  necessary  to  let  h(t)  be  less  than  a  maximum  at  some  times  in 
order  that  it  might  be  higher  "on  the  average"  over  the  given  time  inter- 
val; i.e.,  we  must  find: 


*This  can  be  proved  rather  simply,  and  the  work  is  omitted  here. 
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H(t),  h(t)     Jn        135 


max 


H(t).h(t)n0  ^W^  d^  « 

Let  us  solve  for  c  when  (ll)  is  true  and  h  is  equal  to  its  maximum 
allowable  value  in  region  II: 

h  =  18°   crib"  =  y  180  (2)  &  (15) 


cy+  y  =  cx-f-   lOx 

c  _  i-i JL_  ^  jj  when  h^H  -  180  (20) 

y  -  x 

Suppose  we  pick  an  initial  position  on  the  line  y  =  x        (6) 

and  choose  the  maximum  allowable  value  of  h  for  that  point. 

Let  P0(x0,  y0)  -  (10,  10) 

from  (2),  h  *  i°-  180  «  180 

and  from  (20),  c  *  cx> 

After  travelling  even  a  minute  distance  from  P0(x0,  y0),  h  "^  —  180 

since  x  <  y  as  soon  as  the  ship  leaves  (10,  10)  with  c  =  oo  ;  and  we 

must  pick  a  new  value  for  h.  %•  picking  a  value  for  h  at  P0  slightly 

less  than  its  allowable  value,  we  may  proceed  a  finite  time  before  being 

forced  to  change  h. 

Let      h0  =  160 

then      H  =  20  (assuming  (ll)) 

c  =  ^22  .  10  =  71  from  (U) 

H 

cH 

=  7.89 


180 

,   cH       .   . 

y  =  y  =  7.89  y  from  (10) 

180 
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7.89t 

j 

H  CH  t 

or  in  general  when  £-—  is  constant,  y  =  y0  e  180  (21) 


and        y  -  yQ  =  c(x  -  xQ)  (22) 

From  (ll)  and  (15) »  &  value  for  h  determines  c. 

The  limiting  value  of  y  for  which  we  shall  be  allowed  to  use  this  con- 
stant slope  c  when  h  =  160  will  be  determined  by  the  intersection  of  (22) 
with  a  line  through  the  origin  having  a  slope,  m,  determined  by 

0  <  h  <  .  (7) 

—   —  m 

In  this  case,  h  =  160  and  m  ■  i|°-  =  % 

160   8 

For  a  line  through  the  origin, 

x  =  i  y  (23) 

m 

and  g 

x  =  9  y 

From  (22)  with  Po(l0,10)  and 

y  -  10  =  71(x  -  10) 


xsZ  4-220  .8 
TT  ^  71    9 


71    9  y 
9y  +  6300  =  568y 
y  =  11.27 
From  (21), 

11.27  =  10e7,89tl 

t-,  =  .01515  when  the  value  of  h  must  be  reduced, 
Reducing  h  in  increments  of  10°,  we  arrive  at  the  following  table: 
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p0(*o»yJ 

c 

Hi 

hi 

cHi 
180 

m 

ti»  time  on  leg 
of  voyage  with 
const,  hi 

(10,  10) 

71 

20 

160 

7.89 

9/8 

.01515 

(10.01,  11.27) 

30.5 

40 

140 

6.77 

9/7 

.0202 

(10.03,  12.90) 

17 

60 

120 

5.67 

3/2 

.0290 

(10.13,  15.20) 

10.2 

80 

100 

4.55 

1,8 

.0476 

(10.50,  18.90) 

6.2 

100 

80 

3.44 

2.25 

.0972 

(11.70,  26.40) 

3.5 

120 

60 

2.33 

3.0 

.5150 

(29.20,  87.60) 

3.16 

123 

57 

2.16 

3.16 

oo 

When  c  =  m,  the  two  lines  (mentioned  on  the  preceding  page)  are  paral- 
lel, and  we  maintain  h  =  57  until  T  *  0.  For  any  other  P0  on  the  line 
x  =  y  a  similar  table  to  that  above  may  be  readily  constructed  since  the 
left  column  will  be  made  up  of  simple  multiples  of  the  above  table  and  all 
other  entries  will  be  exactly  the  same  as  those  above.  The  first  column  is 
given  for  two  other  initial  points  as  follows: 

(i;o,~i.o)  (ioo,  ioo) 

(1.001,  1.127)  (100.1,  112.7) 

(1.003,  1.290)  (100.3,  129.0) 

(1.013,  1.520)  (101.3,  152.0) 

(1.050,  1.890)  (105.0,  189.0) 

(1.170,  2.640)  (117.0,  264.0) 

(2.920,  8.760)  (292.0,  876.0) 

The  important  point  to  note  is  that  the  time  on  each  leg  for  the  above 
points  is  the  same  as  that  shown  in  the  original  table. 

We  may  predict  the  result  of  repeating  this  procedure  by  taking  pro- 
gressively smaller  increments  between  successive  values  of  h  used  in  the 
table,  and  our  solution  is: 
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c  ■     y  .  x         for  (x  <  y  <     ^^    x)  (20 

or  (x  <_  y  £-  jflOx) 

and  our  equations  for  h  and  H  are: 

1620^  c  +  1  /     v        /      x 

H  =  7TTo  '      h"  ^Tio   180  to)  4(15) 

Define  a  set  Ila  of  points  from  region  II  such  that  (x  <^  y  5l^-0x) 
for  all  points  in  Ila  and  such  that  this  relation  will  hold  for  no  point 
of  II  which  is  not  in  Ila.  The  above  solution  is  the  optimum  solution  for 
H  and  h  for  all  values  of  x  and  y  in  Ila  for  all  T.  The  optimum  track  for 
a  ship  which  is  initially  at  (x^y^  in  Ila  may  be  obtained  by  solving  the 
following  differential  equation: 

cg.i^Z      (xo<yo<Vl0xo)  (25) 

(lOx  -  y)  dx  -f-  (x  -  y)  dy  =  0  is  of  the  form: 

M  dx-f-  N  dy  =  0  and  the  integrating  factor  is: 


u     = -1 1 «  1 

'  xM  f  yN  10xd  -  xy  +  xy  -  y*        10xe  -  y2 

JP  Mdx  '  jiSTTfz^  »  J  In  (lOx    -  y  )  -  s-^ln  Jjg;* 

2       (  (u  Mdx  X  =  i  zJy_ 1  jgOxJli      f-  2  y^Q  *    \  -±=JL 

^—  HP  MdxJ  -ai5?:yr27H      ^10x-y      [(/lixf  y)*JlOx*- 

p   N  -    JLj  fp  Mdx/     =  R(y)   -  0 
#\  Solution  of  (25)   is: 

i  ln(lOx*  -  /)  -  jjjy  In     A^  =  J  ln(lOx*     -  y*  ) 


y* 


_1_  ln  /Ioxq  -  yc 
2  /16  /l0xo  t  y 


_0_ 

ro 
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or, 


In 


10xfc  -  y' 


1^4  "  7o  J 


^°  [(  /lOx  t  y)  (  \/l5xft  .  y  ) 


(26) 


Equations  (l)  through  (26)  are  shown  in  a  more  general  form  in  Figure  5« 
Solutions  for  other  regions  of  the  ocean  will  be  covered  in  the  next 
chapter. 
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{C/  '  >  '       180  c/zy  \Ctl>tJ  1 80        a,CiCfxi  ISO'  ct,C  rVi- 


0+) 

c  =    -p    -to 

US') 

*s"*f£&J 

(16) 

7               C   i/»     J 
whe*  Hi)   holds. 

(n) 

y  ~    y»   e 

hr   6*y  **J 

(Id) 

y  =  tl 

az-/ 


y 


Y       '#*  y  '**       J  (n)      yzy.e*1*        M     y^yJ 

(S)        0  £    H  5  /SO      (^    sol^e  "S   ^  for     (l~y  ^Xj  f6t  (l<y  <x)  Kb**-* 

ill       o  5  h    -    -prT     Wnl      u  ^  /#o    -  c/±  h*,  ■  >  " 

when  (ll)    Itolc/s.  wt>er>  (ttA)  t>9/</s.  '  7 

<W      h+H=/8o(n*)    (^o)i-(ifo)^1  (zi)    y-y*  =  <=<>*;  d")    i«m<>~(zt).  FIGURE     5 


m*    *,*,     (UA)     (&){''J&t)-'  (E3)        v   ,    «x  C«4        '•«*    "^ 


tl 


CHAPTER  IV 
DYNAMIC  LINEAR  PROGRAMMING,  EXAMPLE  II 

Dr.  Isaacs  introduced  the  mathematical  model  for  the  preceding  prob- 
lem in  a  working  paper  entitled  "A  Problem  in  Programming  Steel  Production 
and  its  Solution  by  Differential  Game  Theory"  '  •  The  following  sections 
represent  this  writer1 s  commentary  and  conclusions  drawn  from  a  study  of 
Dr.  Isaacs*  ideas. 
Part  I,  The  Problem 

The  problem  may  be  stated  as  follows: 
Given ; 
Xi(t)  =  the  number  of  extant  steel  mills  at  time  t. 
*2(t)  =  quantity  of  steel  (or  capital)  available  for  sale  or  use  at  time  t 

(includes  all  unobligated  capital  but  not  stockpiled  steel). 
T  =  a  specified  time  interval  after  which  the  problem  or  game  is  to  termi- 
nate. Henceforth,  to  distinguish  the  two  time  variables,  T  will  be 
referred  to  as  problem  time,  and  t  will  be  called  real  time  when  the 
possibility  of  confusion  exists.  Note  that  T  decreases  as  t  increases. 
0i  =  the  ratio  or  percentage  of  x^  allocated  to  making  new  steel  mills  at 

time  t.  0^  Xj  is  the  amount  so  allocated. 
0?  =  the  ratio  or  percentage  of  X£  devoted  to  making  more  steel.  0^2  ^8 

the  amount  so  allocated. 
1  _  (0]/t"02)  =  *^e  rati-0  or  percentage  of  X2  to  be  stockpiled  at  time  t. 

G.  -  (0t+  02)7  *2  is  the  amount  so  allocated. 
Problem : 

To  find  values  for  01(x1,x2,T)  and  0  (xlfX2,T)  that  will  maximize  x2 
when  T  =  0. 
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The  correspondence  between  the  notation  of  this  chapter  and  Chapter  III 
may  be  indicated  as  follows: 

xx  =  x  02  =-^~  b]_  r  b! 


x2  =  y  al  =  al 


T  -  T 


01  «  —  a2  =  a2  t  =  t 

Assumptions ; 

(1)  x^  is  a  monotonically  increasing  value  such  that  its  rate  of  in- 
crease is  directly  proportional  to  the  amount  of  steel  devoted  to  making 
new  steel  mills.  If  a-^  is  the  constant  of  proportionality: 

*1  =  ^^l3^  ^ 

In  a  generalized  problem  of  course  0  ■  0^(x,,X2,T)  and  02  =  02(xl»x2»T)» 

however,  we  shall  see  that,  in  this  example,  0^  and  02  remain  constant 

over  certain  sub-regions. 

(2)  the  rate  of  increase  or  decrease  of  capital  (or  steel  on  hand)  is 
equal  to  the  algebraic  sum  of  three  rates,  r^,  r2,  and  r  .  r,  is  the  rate 
of  decrease  due  to  expenditures  for  making  new  steel  mills,  r2  is  the  rate 
of  decrease  due  to  expenditures  for  making  new  steel,  and  ro  is  the  gross 
rate  of  increase  due  to  allocating  02r2  caPital  to  raak©  new  steel.  In- 
creasing rates  are  positive  and  decreasing  rates  are  negative. 

a.  r,  is  linearly  dependent  at  any  time  t  on  the  amount  of  capital 
devoted,  at  the  same  time  t,  to  making  new  steel  mills  with  a  proportionality 
constant  of  unity:  r  =  -(0-Xjp) 

b.  r2  is  linearly  dependent  on  the  amount  of  available  capital  de- 
voted, at  the  same  time  t,  to  making  new  steel  with  a  proportionality  con- 
stant of  unity:   r2  =  -  (02x2^ • 
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c.  To  is  linearly  dependent  on  the  amount  of  available  capital 
devoted,  at  the  same  time  t,  to  making  new  steel  with  a  proportionality 
constant  of  a2:   r?  =  -f-  a2(02x2). 

The  problem  requires  that  the  value  of  steel  output  exceed  the  equivalent 
steel  value  of  inputs  required  to  make  the  steel. 

.*.  a2>]L 
Therefore,     x^   =  -(^x^  -  (02x2)  +  a2^2x2) 
or  *2  =  [02(a2-l)  ~  #{]  x2  '  (2) 

(3)  Problem  time  T  remaining  at  any  real  time  t  is  equal  to  the  ini- 
tial problem  time  allowed  minus  the  real  time  t  since  the  problem  started: 

T  =  T0  -  t 
or  T  =  -  1 

(O  Capital  once  allocated  to  one  of  the  three  basic  uses  cannot  later 
be  converted  to  other  uses  (note  1,  page  3»  of  Dr.  Isaacs'  paper).  We  can- 
not allocate  more  than  100^  of  our  available  capital: 

&1>0;       01>O;         02  >  °J     1  -(01+  02)  ~  °       or  0-J  + 02  £l 

(3) 
(5)  ro  must  be  equal  to  or  less  than  byx:^,  (the  product  of  the  number 
of  completed  and  fractions  of  completed  steel  mills  x-i(t)  and  the  maximum 
rate  b,  of  making  steel  per  steel  mill) •  b.  is  a  constant  and  therefore 
independent  of  the  number  of  steel  mills  in  operation,  independent  of  time, 
and  independent  of  the  allocation  of  funds: 

b1x1  ^>  r~  =  a2(02X2)  or  -I*!  ^  02  where  (b->  0) 

a^,  &2»   anc*  b]_  a^e  known,  invariant  quantities. 

From  assumption  (3),  T  =  T0  -  t,  we  see  that  when  an  amount  of  time 
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elapses  equal  to  T0,  the  original  time  interval  given  in  which  we  were  to 
maximize  x2,  then  T  =  0  and  our  problem  ends.  Consider  a  three  dimensional 
Euclidean  space,  £  #  with  axes  x,,  Xg,  and  T  as  shown  in  Figure  6,  poi^e  30. 
If  we  are  given  initial  values  x^,  x2,  and  TQ  (label  this  point  PQ(x1,x2,T  ) 
and  are  told  to  invest  and  reinvest  our  capital  so  as  to  maximize  the  total 
steel  inventory  at  the  moment  when  T  =  0  (or  equivalently,  after  TQ  units 
of  time  have  elapsed  from  problem  starting  time),  we  can  consider  our- 
selves as  guiding  a  point  in  £  starting  at  PQ(x.  ,X2,To)  by  picking  opti- 
mum values  for  0]_  an(*  $2  su^Ject  to  the  restrictions  that  0^  ^>  0, 

0  -f-  tf0  -<.  i  and    ^  >  0O  >  0  as  given  in  the  assumptions  previously 

1    *         a2x2 
listed,  A  combination  of  0,  and  02  is  here  defined  as  a  strategy  (0p02), 

and  we  shall  denote  an  optimal  strategy  by  (0..,02).  Dr.  Isaacs  calls  the 
vector  sum  of  x-^jL,  and  T  for  given  values  of  0,  and  02  a  velocity  vector 
and  calls  the  plane  where  T  =  0  the  terminal  or  capture  surface  S.  After 
considering  all  possible  strategies  subject  to  the  above  restrictions,  we 
must  pick  an  optimum  strategy  for  any  point  (x,  jXg,  T)  which  gives  us  the 
maximum  value  of  x~   when  our  moving  point  (having  passed  through  (x,,x2,T)) 
hits  the  capture  surface,  S.  This  value  of  x>p  is  defined  as  the  payoff, 
V(x_ ,X2,T).  Repeating  this  procedure  for  every  such  point  in  C  is  not 
only  impossible  but  likewise  unnecessary.  Under  the  strong  assumptions 
given  for  this  particular  problem,  it  is  possible  to  divide  C  into  a 
finite  number  of  well-defined  sub-regions  such  that  the  optimal  strategy 
determined  for  a  point  in  any  one  of  the  sub-regions  will  also  be  an  op- 
timal strategy  for  any  other  point  in  that  sub-region.  Dr.  Isaacs  proceeds 
to  show  how  to  determine  the  boundaries  of  these  sub- regions,  the  strate- 
gies applicable  for  each,  and  the  shift  in  strategies  required  as  our 
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moving  point  leaves  one  sub-region  and  enters  another  sub- region  enroute 

to  the  capture  surface. 

In  order  to  simplify  the  discussion,  Dr.  Isaacs'  notation  will  be 

changed  somewhat  for  the  remainder  of  this  paper  as  follows: 

Let  V(x,,x  ,T)  indicate  that  the  payoff  is  a  function  of  x,,x^  and  T, 
Let  V(P;0,02)  indicate  the  payoff  that  will  be  obtained  by  starting 

from  the  point  PCx^x^T)  and  proceeding  according  to  the  strategy  (0i»02), 

where  of  course,  0^  =  0,(xt,x2,T)  and  02  =  02(x-,x2,T). 

Let  V(P;0-]_  ,02  )  indicate  the  payoff  that  will  be  obtained  by  start- 

ing  at  P  and  proceeding  according  to  an  optimal  strategy  (0^  ,02  )• 

Evidently,  V(P;01,02)  <  VCP^*^*)  ,  the  equality  holding  only  when 

(0-,02)  is  optimal.  We  see  that  a  strategy  (0^»02)  must  uniquely  determine 

a  velocity  vector  for  every  point  of  £    •  This  fact  immediately  leads  to 

the  statement  that  a  given  strategy  and  a  given  initial  point  will  uniquely 

determine  all  subsequent  positions  for  our  moving  point. 

v  x     b^ 
From  assumption  (5),  02  f^  -X-=.  =  a"]?    and  from  assumption  {h) ,   02  j£l« 

xl 
Here  we  see  two  upper  bounds  on  02  and  note  that  the  two  are  equivalent 

where  —  *  l£  and  that  for  -=■<  -2.  .  or  Xo*>__L  x_ ,  assumption  (5)  gives 
a2   Xl  a2   xl         a2  1 

the  least  upper  bound;  and  for  -±  ^  _2  or  X2^  —  x. ,  assumption  (k) 
gives  the  least  upper  bound. 

Since  we  are  not  really  interested  in  any  upper  bound  which  is  larger 


than  the  least  upper  bound,  we  may  construct  a  plane,  p>*  containing  the  T 

*1 
axis  and  intersecting  the  plane  T  =  0  in  the  line  x2  =  -=■  x,  and  note  thai 

bn  X  /» 

02  < -  for  X2  values  above  this  plane.  This  portion  of  C  is  labeled 

"  a2x2 


26 


I  in  Figure  6.  Likewise,  ^2  ~  ^  for  x2  values  below  the  given  plane,  and 
this  portion  of  C  i*  labeled  II, 

If  we  plot  the  locus  of  possible  combinations  for  0^  and  02  under  as- 
sumptions (A)  and  (5)  for  regions  I  and  II,  we  have  Figure  7.  In  region  I 
we  may  choose  any  combination  (0^,  00  contained  in  the  hatched  portion  of 
Figure  7a;  and  in  region  II  we  may  choose  any  combination  (0, ,02)  contained 
in  the  hatched  portion  of  Figure  7b, 

Dr.  Isaacs  remarks  that  at  each  point  of  C  »  we  have  at  our  disposal 
a  choice  of  velocity  vectors  whose  totality  comprise  a  pyramid,  the  base 
being  triangular  in  I  and  trapezoidal  in  II,  the  two  merging  continuously 
as  we  cross  the  plane,  P»,  and  he  states  that  we  should  verify  that  none 
of  the  velocity  pyramids  ever  degenerates  into  a  figure  of  fewer  dimensions. 
This  writer  believes  that  these  statements  mean  we  should  verify  that  the 
equations  (l),  (2),   and  (3)  involving  0^  and  02  do  not  make  such  restric- 
tive conditions  on  the  0]_  and  02  that  one  of  the  04  is  determined  as  soon 
as  the  other  is  specified.  This  may  be  tested  by  examining  the  determi- 
nant of  coefficients  of  the  "variables"  0..  ,i02,  an(*  1* 

(1)0X  -r  (1)02        -(1)1  =  0  (3) 

(a1x2)01-f-  (O)02       +  (0)1  -  *x  (1) 

-.(x2)01+  X2(a-1)02    +  (0)1  =  x2  (2) 


11-1 


(aax2) 


0         0 


=  -  x2  ai(a2  -  !) 


— x2  x2^a2  ~  ^  ^ 
If  this  quantity  is  zero,  there  must  be  a  redundancy;  but  since  we  can  see 
that  it  never  vanishes,  we  are  assured  that  0^  and  02  are  not  dependent  on 
each  other. 
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As  previously  mentioned,  our  strategies  can  be  resolved  into  five 
basic  (not  pure)  strategies.  Our  problem  has  been  somewhat  simplified 
since  we  do  not  have  to  work  out  a  solution  for  every  point  in  £  but  can 
resolve  £  into  a  finite  number  of  sub-regions  and  find  which  strategy  is 
optimal  in  each.  So  far  we  have  divided  C.  into  two  regions  I  and  II.  Now 
we  must  examine  the  nature  of  any  surfaces  by  which  we  might  further  divide 
£  into  smaller  sub-regions. 

Because  of  the  simple  linear  form  of  the  kinetic  equations  and  thet 
given  restrictions,  we  may  phrase  a  theorem:  If  more  than  one  optimal 
strategy  will  lead  to  a  value  V  at  the  plane  T  =  0  and  these  routes  are 
representable  by  routes  which  diverge  after  passing  through  a  point,  P, 
all  such  routes  may  be  confined  to  one  continuous,  smooth  surface  defined 
by  the  equation  V(P;0]_  ,02  )  s  const.  This  surface  will  intersect  the 
plane  T  =  0  in  a  half-line  such  that  x-j_  >  0  and  x2  =  VCP;^  ,02*)  *  const. 

If  we  accept  the  preceding  theorem  for  every  point  in  £  ,  we  see  that 
the  solution  to  the  programming  problem  consists  of  a  family  of  surfaces 
filling  £,  without  intersecting,  each  surface  being  the  locus  of  points 
where  V(P;0i  ,02  ")  a  const.  The  velocity  vectors  which  describe  the  path 
of  a  point  governed  by  an  optimal  strategy  are  necessarily  tangent  to  the 
surface  containing  the  moving  point  at  all  times;  and  no  velocity  vector 
can  make  an  angle  of  less  than  90  degrees  with  the  normal  to  the  surface 
at  any  point.  > 

An  important  part  of  the  concept  that  the  payoff  is  a  function  of  x. , 
x2,  and  T  only  is  the  idea  that  the  payoff  is  independent  of  real  time  t 
but  dependent  on  the  problem  time  T  remaining  in  which  to  maximize  x^,. 
Therefore,    *£    -  VL  4*1  +  *L  i&+  &L    <*!   -  n 
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°r         Vx1il+Vx2i2+VT^  =  0 

and   V  (a^*  Xj)  +  VX2  J02*(a2-1)  -  ^"j  Xj  -  VT  =  0  (A) 

since   i^a,^,*/)  ;  £d  =  *i[fcfo-/)-0j;  7--/. 

In  this  example  we  have  five  basic  strategies  represented  by  the  ver- 
tices in  Figures  7a  and  7b,  and  those  vertices  are  labeled  accordingly. 
Dr.  Isaacs  proves  that  the  optimal  strategy  at  any  point  of  £.  must  be  one 
of  these  basic  strategies.  From  equations  (l)  and  (2)  we  may  write  equa- 

tlon U)"  (Zl)  lwa'**&v*Jfa-i-*i$-vTs0 

Let  the  quantity  in  braces  in  equation  (5)  be  represented  by  m^  where 
the  subscript  indicates  which  vertex  of  Figure  7  represents  the  strategy 
being  employed.  To  arrive  at  the  surface  V(,P;0,  ,  0~  )  s  const.,  we  start 
from  the  plane  T  =  0  since  we  know  that  any  such  surface  must  intersect 
T  =  0  in  a  horizontal  line.  By  examining  the  max^Cm^)  for  regions  of  c 
analogous  to  those  described  for  two  dimensions  in  the  preceding  chapter, 
we  may  decide  on  an  optimal  strategy  for  each  region  as  follows: 

(1)  Pick  an  initial  point  xi  =  si»  x2  =  82 

T  «  0 

(2)  Find  the  max^(m^)  for  this  point. 

(3)  Let  T  =  -  t  (thus  reversing  real  time  mathematically)  and  let 

#.*,•    •  #rsfc;  etc. 

O  O  O  a  O  o 

Find       Vx,      j      v*i     j      vT       ,        %'       5  *£.      ?    and  T  at  this  point. 
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{U)   Find  the  solution  to  the  differential  equations  found  in  step  (3). 
(5)  Inspect  this  solution  for  the  smallest  value  of  T  at  which  any  of 
the  following  occurs: 

a*  Another  mj  supercedes  the  m  found  to  be  max  at  the  initial  point. 

b.  The  solution  crosses  the  boundaries  where  "ooi  =  1  or  r^   =  0. 

c.  The  solution  crosses  a  boundary  similar  to  that  given  for  the 
problem  in  the  preceding  chapter  where  the  strategy  is  known  to  change. 

If  b.  occurs,  we  terminate  the  path.  If  a„  or  c.  occurs,  we  take  a  new 
initial  point  where  this  happens  and  repeat  the  above  steps.  Dr.  Isaacs 
proves  that  Vx  ,  Vx  ,  and  V-p  are  continuous  at  transition  surfaces  for  this 
problem.  He  also  performed  the  above  steps  for  a  generalized  problem  and 
obtained  a  general  solution.  His  results  indicate  that  the  results  given 
in  the  preceding  chapter  were  correct  as  far  as  they  went;  but  in  the 
general  case  additional  breakdowns  of  the  given  sub- regions  are  required; 
and,  of  course,  additional  solutions  are  required  for  each  additional  sub- 
region.  Since  the  methodology  should  be  clear  from  the  above  brief  des- 
cription, no  attempt  will  be  made  to  examine  Dr.  Isaacs*  solutions  for  all 
the  sub- regions. 
Part  II.  The  Model. 

A  mathematical  model  can  be  a  useful  tool  when  one  is  seeking  a  quanti- 
tative basis  for  decision;  but  we  must  recognize  where  our  model  falls  short 
of  a  real  situation.  The  methods  used  in  finding  a  solution  to  the  mathe- 
matical model  may  introduce  additional  errors  into  our  results;  and,  con- 
versely, very  precise  mathematical  solutions  are  not  necessarily  required 
for  fairly  crude  models.  Let  us  examine  the  mathematical  model  from  the 
#Dr.  Isaacs  uses  the  boundary  x-%   =  0  vice  x~=  1. 
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preceding  problem  in  three  respects:  (l)  the  nature  of  some  of  the  con- 
stants employed;  (2)  the  form  of  the  kinetic  equations;  and  (3)  the  type 
of  solution  employed. 

The  linear  programming  approach  is  more  suitable  for  a  stable  situation 
than  for  a  dynamic  one.  We  assumed  that  the  rate  of  increase  of  new  steel 
mills  was  linearly  proportional  to  the  amount  devoted  to  making  new  mills 
with  a  proportionality  constant  of  a  •  We  assumed  that  the  rate  of  making 
new  steel  was  linearly  proportional  to  the  amount  devoted  to  making  new 
steel  with  a  constant  of  &2»     We  assumed  that  the  maximum  rate  of  making 
steel  per  steel  mill  was  b    Suppose  we  examine  these  constants.  The  con- 
stants a,  and  a  will  be  quite  different  when  labor  and  raw  materials  are 
plentiful  than  when  most  skilled  labor  and  steel-making  raw  materials  are 
in  short  supply,  as  they  must  become  as  steel  production  increases.  The 
constant  &2   will  also  depend  to  a  large  extent  on  the  capacity  level  at 
which  the  plants  are  operating,  and  is  thus  hard  to  determine  since  capacity 
level  is  always  changing.  The  constants  a  2  and  b^  depend  on  the  impetus  be- 
hind the  movement  to  increase  production  and  also  on  the  overall  production 
effort  since  transportation  facilities,  skilled  labor,  and  other  factors 
become  bottlenecks.  The  constants  which  might  be  derived  from  an  exhaus- 
tive study  of  past  operating  records  would  be,  therefore,  a  very  crude  ap- 
proximation for  projected  use  in  a  dynamic  programming  problem. 

The  kinetic  equations  used  in  these  programming  examples  are  identical, 

except  for  notation,  with  those  used  by  Bellman    in  a  very  similar  prob- 

(2) 
lem.  In  another  problem  by  Bellman   ,  in  which  the  steel  and  machine  tool 

industries  were  assumed  interdependent  with  no  capacity  restraints,  the 

equations  are  more  complicated.  The  usual  assumptions  of  linearity  were 
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maintained,  and  the  solution  again  results  from  a  set  of  homogeneous  dif- 
ferential equations. 

Dr.  Isaacs  used  a  differential  game  theory  solution  to  the  problem  in 
the  preceding  chapter.  A  solution  using  differential  equations  instead  of 
difference  equations  was  introduced  in  this  paper  since  it  is  felt  that  dif- 
ferential equations  provide  a  better  notion  of  behavior  and  are  more  readily 
solved  by  non-mechanical  methods,  than  are  difference  equations.  In  solving 
a  problem,  the  difference  equations  might  be  preferable  for  several  reasons: 
(l)  they  are  amenable  to  digital  computer  operations;  (2)  they  permit  the 
use  of  discrete  decision  times  and  the  use  of  time  lags  for  steel  alloca- 
tions to  produce  results;  and  (3)  they  provide  for  increasing  the  number  of 
steel  mills  in  finite  steps  rather  than  in  a  continuous  manner.  Since 
steel  mills  are  such  large  units  and  since  there  will  always  be  relatively 
few  of  them,  even  at  the  peak  of  the  national  economy,  the  improvement  is 
obvious.  A  third  type  of  solution  would  be  the  standard  perturbation  tech- 
nique leading  to  a  Lagrange  multiplier  formulation. 

Another  outstanding  difficulty  with  the  model  we  have  used  is  that  it 
is  deterministic  instead  of  probabilistic;  and  yet  in  steel  production  we 
must  have  inventory  control  and  thus  probabilistic  dispersion;  we  must  have 
a  safety  allowance  which  increases  as  our  level  of  operations  increases  and 
as  the  penalty  factor  for  depletion  becomes  higher.  The  same  consideration 
applies  in  other  problems  and  should  not  be  ignored. 

Most  of  the  above  remarks  about  mathematical  models  have  been  directed 
at  the  preceding  example,  but  this  does  not  constitute  a  criticism  of  that 
example.  It  merely  emphasizes  some  ground  rules  which  apply  when  we  are 
using  the  model.  Furthermore,  we  are  not  really  interested  in  steel 
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production  at  all  in  this  paper.  We  are  interested  in  an  application  of 
dynamic  linear  programming  and  the  difficulties  and  limitations  associated 
with  it.  The  specific  difficulties  mentioned  have  their  counterparts  in 
other  dynamic  linear  programming  problems. 
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CHAPTER  V 
GAME  THEORY 

As  the  problem  becomes  still  more  general,  we  must  consider  the  effects 
of  possible  counteractions  or  countermeasures  by  opposing  forces:  we  look 
for  minimax  strategies  and  plan  in  terms  of  game  theory.  Here,  we  do  not 
try  to  get  a  numerical  answer  to  a  problem;  but  we  hope  to  get  an  insight 
into  the  nature  of  the  general  problem  and  to  learn  how  to  disregard  poor 
strategies  and  to  pick  some  combination  of  strategies  that  will  put  us  rea- 
sonably "close"  to  a  minimax  solution.  A  classical  example  is  the  Hotspot 

(9) 
game  by  Dr.  Tompkins   ,  which  is  summarized  below. 

Consider  two  opposing  forces  made  up  of  like  vessels,  Blue  and  Orange. 
Each  is  subject  to  damage  from  opposing  ships  and  also  from  chance  causes 
such  as  navigation  hazards  and  mines,  but  only  when  in  a  certain  area  called 
Hotspot.  The  action  proceeds  in  a  sequence  of  discrete  moves  of  indefinite 
extent.  Initially,  Blue  has  mi  units  and  Orange  has  ^  -,   units;  and  at  the 
beginning  of  the  k-th  move  Blue  has  m^  and  Orange  has  u  ^  units,  where  these 
numbers  are  determined  by  the  results  of  the  preceding  action.  The  battle 
is  for  control  of  Hotspot,  and  strategic  value  is  assigned  in  terms  of  gain 
or  loss  to  Blue  as  follows :  if  Blue  remains  in  Hotspot  and  Orange  with- 
draws, Blue  gains  an  amount  A  strategically;  if  Blue  withdraws  and  Orange 
remains,  Blue  loses  an  amount  B  strategically.  If  both  withdraw,  Blue's 
strategic  situation  neither  gains  nor  loses  value.  Each  move  consists  of 
each  contestant  assigning  a  number  of  his  units  to  fight  at  Hotspot.  This 
number  may  be  zero  or  any  portion  of  his  available  forces.  For  the  kth  move, 
Blue  must  assign  a  number  a,  of  units  to  action,  where  a  is  an  integer  with 
0  ^  a^  ^  m.  ,  ra^  being  the  total  force  available;  and  Orange  must  assign  a 
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number  oC^  of  units  to  action,  where  c<  k  is  an  integer  with  0  j£_<X,  —  Mk* 
The  action  terminates  if  either  a,  or  od ,  or  both  are  zero.  If  this  happens 
for  some  k  (as  it  must  sooner  or  later  because  of  losses),  then  this  is  the 
last  move  and  Blue,  Orange,  or  both  will  have  withdrawn  with  the  strategic 
consequences  listed  above.  Intelligence  concerning  forces  available  and 
evaluations  assigned  is  perfect;  intelligence  concerning  plans  is  zero. 
That  is,  before  assigning  forces  for  the  kth  move  both  Blue  and  Orange  know 
both  the  numbers  m.  and  p  ^  and  the  constants  A  and  B;  and  they  also  know 
loss  probabilities  and  some  other  constants  listed  below.  At  the  beginning 
of  a  move  the  loss  can  be  predicted  only  probabilistically.  That  is,  there 
is  a  known  function  P(p;7T  :  r^p)   defined  for  all  pairs  of  non-negative 
integers  r, p     and  for  all  non-negative  integral  values  p  and  TT  not  ex- 
ceeding r  and  p       respectively.  Naturally,  the  sum  of  all  values  of  P  is 
one.  P  is  the  joint  probability  of  p  losses  to  Blue  and  TT     losses  to  Orange 
if  Blue  commits  r  of  his  forces  and  Orange  commits  p  of  his  forces  to  ac- 
tion in  Hotspot.  In  addition  to  the  strategic  value  of  Hot spot,  Blue  loses 
one  unit  for  each  of  his  units  lost  in  battle  and  gains  <T  units  for  each 
unit  lost  by  Orange.  Orange  knows  the  values  assigned,  and  his  ambition  is 
to  oppose  as  strongly  as  possible  Blue's  attempt  to  gain  in  terms  of  these 
values. 

A  and  B  are  part  of  the  evaluation  of  the  problem  by  Blue  and  Orange; 
they  give  not  only  a  statement  of  the  appropriate  effect  desired,  but  a 
statement  of  how  greatly  the  effect  is  desired.  Blue's  plan  will  be  one 
which  assures  least  loss  from  Orange's  actions:  Blue  considers  every  possi- 
ble plan  of  action,  considers  Orange's  best  reply  to  each  plan,  and  then  se- 
lects a  plan  which  maximizes  Blue's  expectation  against  Orange's  best  reply. 
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Blue  may  assume  at  the  time  of  the  first  move  an  expected  gain  or  loss  from 
the  action;  and  that  expectancy  constitutes  the  strategic  or  the  military 
value  of  his  position  at  that  time. 

The  problem  of  computing  a  value  function  is  to  assign  a  value  V(m,u  ) 
to  Blue's  expected  net  gain  if  Blue  has  m  units  and  Orange  has  u  units,  and 
if  each  moves  as  well  as  possible.  Dr.  Tompkins  computes  a  table  of  values 
for  V  for  a  simple  case  to  show  his  methodology,  but  no  attempt  will  be 
made  to  summarize  that  part  of  his  results  here.  The  important  feature  of 
the  table  that  is  of  interest  here  is  that  any  entry  in  the  table  may  be 
computed  from  entries  which  precede  it;  and  thus,  the  table  may  be  con- 
structed progressively  from  one  basic  entry  if  values  are  given  for  A,  B, 
and  other  conditions  are  as  previously  stated.  Dr.  Tompkins  proves  the 
existence  of  the  Evaluation  Function:  V(m,  pi  ). 

The  Hot spot  game  was  introduced  here  to  show  a  method  for  arriving  at  a 
quantitative  basis  for  decision  when  we  are  faced  with  an  apparent  hodge- 
podge of  imponderables.  A  successful  military  commander  is  successful  be- 
cause he  has  a  wealth  --of  experience  and  training  which  help  him  "intuit" 
an  effective  solution  from  a  maze  of  tactical  information.  How  much  more 
effective  should  military  planning  become  if  the  commander  and  his  staff 
are  provided  with  a  framework  of  evaluation  into  which  values  based  on  com- 
mand experience  can  be  inserted  for  complex  analyses  of  their  implications! 
Naturally,  a  mathematical  model  is  not  an  exact  picture  of  any  real  situa- 
tion and  military  planners  must  practise  the  use  of  such  tables  in  peace- 
time exercises  in  order  to  work  effectively  with  the  new  tool  provided. 
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APPENDIX 

PROBLEM  EXAMPLES 

1,  This  problem  is  the  same  as  that  stated  on  the  bottom  of  page  1. 
Notation  is  the  same  as  that  used  in  Chapter  II. 
Let  x^  =  coal  consumed  by  furnace  #1, 
x2  *  coal  consumed  by  furnace  #2. 
100  tons  ■  total  coal  available  initially  (x^-h   x2  =  100). 
f.  (x]_)  =  total  steel  output  from  furnace  #1  from  x.  tons  coal. 
^2^x2^  =  total  steel  output  from  furnace  #2  from  x£  tons  coal. 
EU^)  =  f1(x1)  -f  f2(x2) 
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A  plot  from  these  values  shows  clearly  that  only  furnace  number  2  should  be 
operated  in  this  example. 

2.  This  problem  is  the  sane  as  tliat  stated  on  the  bottom  of  page  three  with 
the  same  assumed  values  as  used  in  Problem  1.  From  the  values  obtained  in 
Problem  1,  the  following  table  is  obtained: 


3fc* 


All  coal  to 
furnace  #1 

One  barge  to 

each  furnace 

All  coal  to 
furnace  02 


Strategies 

til 

Bomb  to 
fact  on-  #1 

© 

25 

© 

20 

© 


100 


tn 

Bonb  to 
factor?,   #2 

85 

25 
20 


Table  of  Possible  Occurrences  Showing  r Steel  Production  for  Each 
Possible  Case. 

To  obtain  the  optimum  strategy  for  the  steel  executive  and  for  the  enemy, 

the  following  plots  are  constructed  as  explained  by  reference  (4)  • 


too 


loo 


These  plots  show  that  strategy  (D  should  be  dropped  completely,  and  that 
strategies  (l)   and  (3)   should  be  used  as  decided  by  a  chance  mechanism 
weighted  4/7  to  3/7  and  strategies  jTJ  and  \2\   decided  by  a  mechanism 
weighted  7/15  to  6/l5»  The  expected  value  of  steel  production  for  these 
values  then  would  be  56  units. 
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c  -     y  .  x         for   (x  <  y  <     1+n-    x) 
or   (x  <   y  £-  jflOx) 

and  our  equations  for  h  and  H  are: 

1620  c  +  1 

H  =  cTlO   5         h  "     cTlO     180 


(24) 


(13)  &  (15) 


Define  a  set  Ila  of  points  from  region  II  such  that  (x  <_  y  5:^-0x) 
for  all  points  in  Ila  and  such  that  this  relation  will  hold  for  no  point 
of  II  which  is  not  in  Ila.  The  above  solution  is  the  optimum  solution  for 
H  and  h  for  all  values  of  x  and  y  in  Ila  for  all  T.  The  optimum  track  for 
a  ship  which  is  initially  at  (x^y^  in  Ila  may  be  obtained  by  solving  the 
following  differential  equation: 


o-g-2*^      (xo<yo<fL0xo) 


(25) 


(lOx  -  y)  dx  -f-  (x  -  y)  dy  =  0  is  of  the  form: 

M  dx-f-  N  dy  =  0  and  the  integrating  factor  is: 


P  ' 


xM  t  yN 


10xd  -  xy  +  xy  -  y2-        10xe  -  yz 


/?  «<* -  JiS^> -  *  *>  ^  -^- rfc* fef 


x  -  y 


r)       \    C.  ,  xia     (   -  1  -  2y  1  yiOxf-  P-  2^10  x 

x)r Mdxf  ^sfcy^Tw  h^  L^xT7)^iox*. 


p   N  -    ]L-\  j p  Mdx/     =  R(y)  -  0 
Solution  of  (25)  is: 


i  lndO,'  -  /)  -  ^=3-  In    ^ff-  =  J  ln(10*<    -  r»  ) 


-       1        In    vfI5xo  "  y' 

"  2  /io         /iox0  t  y( 
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